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ABSTRACT 

This paper examines practical and theoretical 
problems and issues that arose during evaluation of an adventure 
program for young offenders. During 1989-93, the Sail Training 
Association and the Humberside (England) Probation Service 
collaborated on a project in which probation clients aged 17-25 made 
sailing voyages across the North Sea or around the coast of Britain. 
Although the underlying "treatment theory" was never fully 
articulated, it revolved around the notion of the personal 
developmental benefits to be derived by young adult offenders from a 
challenging physical experience.- Evaluation included five elements: 
(1) reconviction rates after voyage completion; (2) changes in 
offender self-perceptions based on an adapted problem checklist 
administered before and after voyage; (3) ship captains' reports; (A) 
postvoyage client se 1 f -evaluat i ons ; and (5) probation officers' 
survey. Reconviction rates of voyagers were similar to those cf 
matched probation controls. Administration of the checklist 
encountered many practical difficulties and was abandoned. 
Nevertheless, ship captains' reports, client self-evaluations, and 
the probation officer survey were very positive and supportive of the 
program. Questions to consider when conducting such evaluations 
include distinguishing betwe-^n indiv^' dual-level and system-level 
objectives, linking evaluation closely to declared project aims, and 
the need for parallel evaluations of outcomes and processes. (SV) 
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I should make It clear at the outset that I shall be approaching this topic today as an 
academic cnminologist. and not as a practitioner - or as a polilician! I was invited^ 

rant^^l^'-^''^'"^,"^"^^^^^^^^^ °f iuvolvemenrwi h col leagues a° 

Hu l University, in an evaluation of Humberside Probation Service's Sail Training 
Project, which has been sponsored by the Rank Foundation for the last 4 years I Sb^ 
ev'ZaSLTS. '""'"- the practical and theoretical problems experienced in oS 
evaluation of this project to identify a number of issues and lessons that might have 'vider 
offended" evaluation of other similar outdoor pursuits projects for young 

DracSeS' k one nf fn"^ """^'i' ' for criminologists and criminal justice 

SXTdir? wh^^^^^ ""T raised about "What works?" 

witn ottenders - whether those who are in prison or those for whom alternative 

of offend ng In this climate it is more than ever important to be able to show 
conclusively that mnovative outdoor pursuits and adventure progi^es prSwde a 4^^^^^ 
^SSrf. """"^ effccyve method of dealing with offenders thS custodiTrother^^^^^^^^ 
custodial alternatives. To those who ask "Why should we be bothered with evalua™- 
v lTlZv^^ ^""'^ professional concerns to demonst^te the ouTme o^ 

fmnn^ H ' ^owever niuch you are personally convinced of its value, but perhaps more 
importantly to provide hard evidence for the public, for politicians and poteS fuS 
of the crime reduction effects of such projects dna potential lunders 

Humberside Sail Tk-aining Project 

In 1989. the Rank Foundation agreed to fund up to 40 places a year for 3 years on Sail 

St'hm^ r^^^^ ^ n ' ^^^^l' °" schooners Malcolm Miller and Sir Winston 
Churchill, and are usually across the North Sea or round tlie coast of Britam 
was aimed at the more serious offenders, at risk of custodial sentencerXlthougS 
underlying "treatment theory" or rationale of the scheme was never fully S iculated U 
revolved around the notion of the personal benefits to be derived by young adi 

~i:rd^set^^^^^^^^ P« -h--^ seff-SK 
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In the event, for a variety of reasons (including the Prison Service's unwillingness to 
allow temporary release from Young Offender Institutions to enable trainees to undertake 
voyages which would involve visiting foreign ports) the number of referrals to the 
scheme was lower than anticipated, so that even after the Rank Foundation agreed to 
extend the scheme for a fourth year, a total of only 60 offenders went on the STA 
voyages between October 1989 and November 1993, [Note: Of the 140 referrals to the 
scheme in the four years, only 11 were rejected outright; but of the rest who were 
selected, over half withdrew or were withdrawn by their supervising probation officers 
before taking up their places on a voyage.] 

There were several elements in the evaluation, which was originally intended to be based 
upon analyses of: 

• reconviction rates for up to 2 years after completion of the voyage; 

• changes in the offenders* self-perceptions, etc, by use of an adapted "problem 
checWist" to be administered before the voyage, and then 2 months and 12 months 
after its completion; 

• reports from the Captain or Watch Officer; 

• clients* self-evaluation of the benefits derived from the voyage. 

In order for meaningful conclusions to be able to be drawn from the reconviction 
statistics and from the results of the problem check-lists, it was important to identify a 
control group of probation clients who matched those who went on the voyages in as 
many relevant factors as possible, eg age, offence, previous convictions, length of 
supervision order and so on. The main aim of the evaluation was thus to sec whether, or 
to what extent, the voyage experience resulted in a reduced level of reoffending when 
compared to a control group of similar young adult offenders on probation in 
Humberside. In addition, it was intended to see whether participation in the STA scheme 
had ai^y significant effect upon offenders* attitudes and self-perceptions (as measured by 
the problem check-list), to see how they coped with the demands of the voyage itself, as 
reflected in the reports of the captain, and to monitor their own views and those of their 
supervising probation officers about what they might have gained from the experience. 

Reconvictions. The initial results of the reconviction data analysis were as follows: 

• out of the 16 voyagers in Year 1, 7 (44 per cent) had further court appearances 
resulting in convictions in the 12 months after the voyage, although usually only 
one such appearance; 

• out of the 16 probation clients in the Year 1 control group, 6 (38 per cent) had 
further court appearances, averaging two each; 

• in Year 2, out of 9 voyagers for whom a control client had been identified, 4 (44 
per cent) had further convictions - normally just one court appearance; 

• whereas 5 (56 per cent) of the control group had further court appearances, 
averaging 2 each. 



Thus, in the first two years of the STA project, for which relatively complete control 
group information was available, there was little significant or systematic difference in 
the proportions who reoffended, although the control g^oup clients tended to reoffend 
rather more frequentiy in the 12 month period. 



In Year 3 of the project, as a decision had been taken by the Probation Service to 
discontinue the use of a control group for the purpose of the administration of the check- 
list, no clients were identified for the purposes of reconviction data. However, police 
statistics showed that of the 17 clients who went on Sail Training voyagers, 7 (41 per 
cent) were reconvicted - which was a very similar proportion as in the first two years. 

It seems clear, therefore, that approximately 4 out of 10 of those who went on the 
voyages were reconvicted in the 12 months after the voyage - but this was a similar rate 
of reoffending as that found among matched probation clients, although the latter tended 
to average rather more couit appearances than the voyagers. 

Problem check-list. The administration of the problem check-list, to monitor any changes 
in clients' attitudes and self-perceptions, was dogged by administrative and other 
difficulties almost from the start. In the first year, all those who went on STA voyages 
completed a check-list before the voyage, but only 56 per cent completed a check-list 2 
months after the voyage, with just 2 out of the 16 voyagers completing the 12 month 
check-list. In the second year (when the administration of the project ran into particular 
problems following the departure of a key worker at Probation Service Headquarters) 
only half the voyagers completed the first check-list, prior to going on the voyage. 

In the light of the practical difficulties encountered in achieving a satisfactory check-list 
completion rate, and particularly because of the difficulties and obstacles surrounding the 
completion of check-lists by control clients, which led to a decision to discontinue the 
control group check-list, it was decided that little of real value was likely to emerge from 
such a diluted exercise as remained, so that the check-lists were not completed in the 
third and fourth years of the project, thereby depriving the evaluation of potentially 
valuable data to put alongside the rather equivocal results of the reconviction data 
analysis. 

However, to assess the potential value of this evaluation instrument we carried out an 
analysis of 4 matched pairs of voyagers and control clients from Year 1 for whom we had 
a pre-voyage and a 2-month post-voyage check-list. From this analysis there were some 
indications that the Sail Training experience helped some of the young adult offenders to 
become more aware of their own behaviour, particularly in relation to discipline. There 
was also sc -^e indication that they found it easier to mix with people after the voyage 
although, i" haps paradoxically, some of them were also more conscious of their 
identities as offenders. On a further positive note, all of the clients after the voyage 
disagreed with the statement that they were dull or uninteresting compared to hAf who 
had agreed with that before the voyage. None of the control group felt that they were 
dull or uninteresting before or after the voyage! Perhaps rather surprisingly, there was 
relatively little change in the clients' self-confidence after the voyage - indeed, the two 
who did not feel confident prior to the voyage also did not afterwards. . Three of the four 
voyagers agreed more strongly in the post-voyage check-list that they liked 
responsibilities, although one client gave a more negative response. 

The results of this analysis of checklists are difficult to interpret in view of the small 
numbers involved. However, the fact that the results do reveal certain positive changes 
suggests that the systematic use of this sort of checklist might be of value in future 
evaluations. Whilst there is some evidence from these findings that the voyages may be 
character building", there are also hints that for some offenders the experience may bring 
to light some of their limitations (eg ability to "stick at things") in a way that might 
decrease their self-confidence or damage their self-image if not properly handled by 
supervisors. 



55 



the valul of L recoSSSr rtf ,„7f°'°iS"'"' «'»f<!'«n'^e of reducing 
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enabled them to make new friends and work as a member of a team; 
increased their self-confidence; 

• provided them with opportunities to travel abroad; 

• gave them a chance to learn new skills. 

UmttlutSt ha'S Sedfhem'?? ""y'^' ">^* 'h™. 

4-vear durarion tS^^ fic I ^^^^ ^^^^"^s to the STA project during its 




• to broaden the horizons 

• enhance self-respect and self-confidence 

• to recognise the claims of other people 

• to provide discipline, and push them to new limits. 

JSn^t' P^'^'P'' "^'^ "° ^^^^"^^ °f Ae aim or likelihood of reducing 

fo conckde^bv^'t^L ^ T'^i" ^"^"^'^ ^™P^^^ °"^door puisuits activities I S 
Process and pitfalls of evaluation 

the individual or sS level Knwfh^^ make comparative statements, at either 
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For many purposes, and many audiences, the primary or exclusive interest is in the 
evaluation of outcomes. Reviews of such evaluations in the field of criminal justice, 
sentencing and the effectiveness of penal measures in the 1970s - first in the United 
States and then in the UK - were very pessimistic, suggesting that "Nothing works". 
Subsequent research and "evaluations of evaluations"(!) has served to modify the 
overwhelmingly negative messages coming from these first surveys, but it has to be said 
that the evidence for the crime-reduction outcome of outdoor pursuits and adventure 
based projects is very limited indeed, at this stage. Moreover, it is often very difficult to 
interpret the apparently negative research findings, partly because they have not included 
any parallel evaluation of the processes involved, which might go some way towards 
explaining the reasons for apparent failures of projects to achieve the desired results. 

Another recognised element in criminological evaluation studies is what has been termed 
tlie "interaction" effect, whereby positive effects of a scheme on some offenders may be 
cancelled out, and thereby "masked", by comparable negative effects on others in the 
scheme. 



A more fundamental possibility that might explain the relative failure (if such it be) of 
outdoor pursuits and adventure based schemes in significantly reducing offending is that 
they are based on underlying (and usually implicit rather than explicit) theories of 
offending behaviour of an individualistic kind that are no longer supported by wider 
criminological research. Arguably the main trends in criminological theory in the last 10 
to 15 years have emphasised the important role of the immediate environment in which 
young people grow up and the situational contexts and opportunities within which crime 
is likely to thrive. In other words, there is no longer the unchallenged faith in the ability 
of inner change to affect the behaviour and resist the challenges posed by the 
environment, and its social and economic pressures. Even if adventure based experiential 
learning has great power to change a person's attitudes and behrviour, it has increasingly 
to compete against a world of almost irresistible pressures to join in deviant and 
delinquent activities in pursuit of individual or collective goals or satisfactions. 

Finally, among the other lessons for the successful introduction and evaluation of 
innovative schemes for offenders that emerge from our experience in Humberside, is the 
need to recognise that Mew schemes have to be "owned" by the service or professions that 
have to deliver - from the senior managers down to the grass-roots practitioners and 
project administrators. The management of innovation requires commitment and 
resources of both people and money to have any real chance of succeeding. There has to 
be shared ownership and agreement on aims and objectives - including, hopefully, a 
recognition of the need to monitor and evaluate the work that is being done. 



Evaluation and communication 

Who or what is evaluation for? 

It is to provide evidence to persuade other professional workers and politicians at local 
and central levels that the work you do is effective. It is to persuade potential funders 
that such schemes are worth investing their money in, for the future of young people and 
the communities of which they are a part. It is to give the workers confirmation of the 
value of what they are doing, not just at the level of faith and "gut feeling" but with 
objective evidence of achievements. 



58 



Changing people*s behaviour is a complex matter - particularly young people whose 
early life experiences have often been damaging and traumatic. Cnme and delinquency 
are a product both of the individual and his or her current environment, with its powerful 
social, economic and personal influences. An apparent failure to show the crime 
reduction effects of schemes should not always be accepted at face value, as there are 
many factors that can affect the methodology and thereby the results. The hope for the 
future lies in harnessing a shared faith in the value of what is being done to the difficult 
but vital task of evaluation to persuade others of this value. 
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